Prioritized Sweeping Neural DynaQ with Multiple Predecessors, and Hippocampal Replays

During sleep and awake rest, the hippocampus replays sequences of place cells that have been activated during prior experiences. These have been interpreted as a memory consolidation process, but recent results suggest a possible interpretation in terms of reinforcement learning. The Dyna reinforcement learning algorithms use off-line replays to improve learning. Under limited replay budget, a prioritized sweeping approach, which requires a model of the transitions to the predecessors, can be used to improve performance. We investigate whether such algorithms can explain the experimentally observed replays. We propose a neural network version of prioritized sweeping Q-learning, for which we developed a growing multiple expert algorithm, able to cope with multiple predecessors. The resulting architecture is able to improve the learning of simulated agents confronted to a navigation task. We predict that, in animals, learning the world model should occur during rest periods, and that the corresponding replays should be shuffled.Comment: Living Machines 2018 (Paris, France

Parasites of the Deep-Sea Smelt Bathylagus euryops (Argentiniformes: Microstomatidae) from the Charlie-Gibbs Fracture Zone (CGFZ)

The deep-sea smelt Bathylagus euryops, caught in July 2004 at the Charlie-Gibbs Fracture Zone (CGFZ) of the Mid-Atlantic Ridge (North Atlantic), was studied for metazoan parasites and diet composition. A total of 86 specimens with standard lengths between 6.4 and 22.1 cm (mean 13.6 cm) were examined. The parasite fauna consisted of five species: three Digenea, one Cestoda and one Nematoda. The predominant parasites were Lecithaster sp. (Digenea) and an unidentified bothriocephalidean cestode. The only nematode, Anisakis sp., occurred with a low prevalence. Bathylagus euryops at CGFZ serves as final host for the three digeneans, and as intermediate host for the cestodes and Anisakis sp. Stomach content analysis revealed a mesozooplankton crustacean diet, while 95.3% of the stomachs contained unidentified tissue

Bayesian optimization for materials design

We introduce Bayesian optimization, a technique developed for optimizing time-consuming engineering simulations and for fitting machine learning models on large datasets. Bayesian optimization guides the choice of experiments during materials design and discovery to find good material designs in as few experiments as possible. We focus on the case when materials designs are parameterized by a low-dimensional vector. Bayesian optimization is built on a statistical technique called Gaussian process regression, which allows predicting the performance of a new design based on previously tested designs. After providing a detailed introduction to Gaussian process regression, we introduce two Bayesian optimization methods: expected improvement, for design problems with noise-free evaluations; and the knowledge-gradient method, which generalizes expected improvement and may be used in design problems with noisy evaluations. Both methods are derived using a value-of-information analysis, and enjoy one-step Bayes-optimality

UK Breastfeeding Helpline support: An investigation of influences upon satisfaction

Background Incentive or reward schemes are becoming increasingly popular to motivate healthy lifestyle behaviours. In this paper, insights from a qualitative and descriptive study to investigate the uptake, impact and meanings of a breastfeeding incentive intervention integrated into an existing peer support programme (Star Buddies) are reported. The Star Buddies service employs breastfeeding peer supporters to support women across the ante-natal, intra-partum and post-partum period. Methods In a disadvantaged area of North West England, women initiating breastfeeding were recruited by peer supporters on the postnatal ward or soon after hospital discharge to participate in an 8 week incentive (gifts and vouchers) and breastfeeding peer supporter intervention. In-depth interviews were conducted with 26 women participants who engaged with the incentive intervention, and a focus group was held with the 4 community peer supporters who delivered the intervention. Descriptive analysis of routinely collected data for peer supporter contacts and breastfeeding outcomes before and after the incentive intervention triangulated and retrospectively provided the context for the qualitative thematic analysis. Results A global theme emerged of 'incentives as connectors', with two sub-themes of 'facilitating connections' and 'facilitating relationships and wellbeing'. The incentives were linked to discussion themes and gift giving facilitated peer supporter access for proactive weekly home visits to support women. Regular face to face contacts enabled meaningful relationships and new connections within and between the women, families, peer supporters and care providers to be formed and sustained. Participants in the incentive scheme received more home visits and total contact time with peer supporters compared to women before the incentive intervention. Full participation levels and breastfeeding rates at 6-8 weeks were similar for women before and after the incentive intervention. Conclusion The findings suggest that whilst the provision of incentives might not influence women's intentions or motivations to breastfeed, the connections forged provided psycho-social benefits for both programme users and peer supporters

Mobile phone apps for behavioral interventions for at-risk drinkers in Australia: literature Review

Background: The mobile technology era has ushered in the use of mobile phone apps for behavioral intervention for at-risk drinkers. Objective: Our objective was to review recent research relevant to mobile phone apps that can be used for behavioral intervention for at-risk drinkers in Australia. Methods: The inclusion criteria for this review were articles published in peer-reviewed journals from 2001 to 2017 with use of the search terms "smartphone application," "alcohol," "substance," "behavioural intervention," "electronic health," and "mobile health." Results: In total, we identified 103 abstracts, screened 90 articles, and assessed 50 full-text articles that fit the inclusion criteria for eligibility. We included 19 articles in this review. Conclusions: This review highlighted the paucity of evidence-based and empirically validated research into effective mobile phone apps that can be used for behavioral interventions with at-risk drinkers in Australia

Carbon storage and DNA absorption in allophanic soils and paleosols

Andisols and andic paleosols dominated by the nanocrystalline mineral allophane sequester large amounts of carbon (C), attributable mainly to its chemical bonding with charged hydroxyl groups on the surface of allophane together with its physical protection in nanopores within and between allophane nanoaggregates. C near-edge X-ray absorption fine structure (NEXAFS) spectra for a New Zealand Andisol (Tirau series) showed that the organic matter (OM) mainly comprises quinonic, aromatic, aliphatic, and carboxylic C. In different buried horizons from several other Andisols, C contents varied but the C species were similar, attributable to pedogenic processes operating during developmental upbuilding, downward leaching, or both. The presence of OM in natural allophanic soils weakened the adsorption of DNA on clay; an adsorption isotherm experiment involving humic acid (HA) showed that HA-free synthetic allophane adsorbed seven times more DNA than HA-rich synthetic allophane. Phosphorus X-ray absorption near-edge structure (XANES) spectra for salmonsperm DNA and DNA-clay complexes indicated that DNA was bound to the allophane clay through the phosphate group, but it is not clear if DNA was chemically bound to the surface of the allophane or to OM, or both. We plan more experiments to investigate interactions among DNA, allophane (natural and synthetic), and OM. Because DNA shows a high affinity to allophane, we are studying the potential to reconstruct late Quaternary palaeoenvironments by attempting to extract and characterise ancient DNA from allophanic paleosol

Reinforcement learning or active inference?

This paper questions the need for reinforcement learning or control theory when optimising behaviour. We show that it is fairly simple to teach an agent complicated and adaptive behaviours using a free-energy formulation of perception. In this formulation, agents adjust their internal states and sampling of the environment to minimize their free-energy. Such agents learn causal structure in the environment and sample it in an adaptive and self-supervised fashion. This results in behavioural policies that reproduce those optimised by reinforcement learning and dynamic programming. Critically, we do not need to invoke the notion of reward, value or utility. We illustrate these points by solving a benchmark problem in dynamic programming; namely the mountain-car problem, using active perception or inference under the free-energy principle. The ensuing proof-of-concept may be important because the free-energy formulation furnishes a unified account of both action and perception and may speak to a reappraisal of the role of dopamine in the brain

A comparative study of Tam3 and Ac transposition in transgenic tobacco and petunia plants

Transposition of the Anthirrinum majus Tam3 element and the Zea mays Ac element has been monitored in petunia and tobacco plants. Plant vectors were constructed with the transposable elements cloned into the leader sequence of a marker gene. Agrobacterium tumefaciens-mediated leaf disc transformation was used to introduce the transposable element constructs into plant cells. In transgenic plants, excision of the transposable element restores gene expression and results in a clearly distinguishable phenotype. Based on restored expression of the hygromycin phosphotransferase II (HPTII) gene, we established that Tam3 excises in 30% of the transformed petunia plants and in 60% of the transformed tobacco plants. Ac excises from the HPTII gene with comparable frequencies (30%) in both plant species. When the β-glucuronidase (GUS) gene was used to detect transposition of Tam3, a significantly lower excision frequency (13%) was found in both plant species. It could be shown that deletion of parts of the transposable elements Tam3 and Ac, removing either one of the terminal inverted repeats (TIR) or part of the presumptive transposase coding region, abolished the excision from the marker genes. This demonstrates that excision of the transposable element Tam3 in heterologous plant species, as documented for the autonomous element Ac, also depends on both properties. Southern blot hybridization shows the expected excision pattern and the reintegration of Tam3 and Ac elements into the genome of tobacco plants.

Unintended Consequences of Incentive Provision for Behaviour Change and Maintenance around Childbirth

Financial (positive or negative) and non-financial incentives or rewards are increasingly used in attempts to influence health behaviours. While unintended consequences of incentive provision are discussed in the literature, evidence syntheses did not identify any primary research with the aim of investigating unintended consequences of incentive interventions for lifestyle behaviour change. Our objective was to investigate perceived positive and negative unintended consequences of incentive provision for a shortlist of seven promising incentive strategies for smoking cessation in pregnancy and breastfeeding. A multi-disciplinary, mixed-methods approach included involving two service-user mother and baby groups from disadvantaged areas with experience of the target behaviours as study co-investigators. Systematic reviews informed the shortlist of incentive strategies. Qualitative semi-structured interviews and a web-based survey of health professionals asked open questions on positive and negative consequences of incentives. The participants from three UK regions were a diverse sample with and without direct experience of incentive interventions: 88 pregnant women/recent mothers/partners/family members; 53 service providers; 24 experts/decision makers and interactive discussions with 63 conference attendees. Maternity and early years health professionals (n = 497) including doctors, midwives, health visitors, public health and related staff participated in the survey. Qualitative analysis identified ethical, political, cultural, social and psychological implications of incentive delivery at population and individual levels. Four key themes emerged: how incentives can address or create inequalities; enhance or diminish intrinsic motivation and wellbeing; have a positive or negative effect on relationships with others within personal networks or health providers; and can impact on health systems and resources by raising awareness and directing service delivery, but may be detrimental to other health care areas. Financial incentives are controversial and generated emotive and oppositional responses. The planning, design and delivery of future incentive interventions should evaluate unexpected consequences to inform the evidence for effectiveness, cost-effectiveness and future implementation

Estimation of proteinuria as a predictor of complications of pre-eclampsia: a systematic review

Background Proteinuria is one of the essential criteria for the clinical diagnosis of pre-eclampsia. Increasing levels of proteinuria is considered to be associated with adverse maternal and fetal outcomes. We aim to determine the accuracy with which the amount of proteinuria predicts maternal and fetal complications in women with pre-eclampsia by systematic quantitative review of test accuracy studies. Methods We conducted electronic searches in MEDLINE (1951 to 2007), EMBASE (1980 to 2007), the Cochrane Library (2007) and the MEDION database to identify relevant articles and hand-search of selected specialist journals and reference lists of articles. There were no language restrictions for any of these searches. Two reviewers independently selected those articles in which the accuracy of proteinuria estimate was evaluated to predict maternal and fetal complications of pre-eclampsia. Data were extracted on study characteristics, quality and accuracy to construct 2 × 2 tables with maternal and fetal complications as reference standards. Results Sixteen primary articles with a total of 6749 women met the selection criteria with levels of proteinuria estimated by urine dipstick, 24-hour urine proteinuria or urine protein:creatinine ratio as a predictor of complications of pre-eclampsia. All 10 studies predicting maternal outcomes showed that proteinuria is a poor predictor of maternal complications in women with pre-eclampsia. Seventeen studies used laboratory analysis and eight studies bedside analysis to assess the accuracy of proteinuria in predicting fetal and neonatal complications. Summary likelihood ratios of positive and negative tests for the threshold level of 5 g/24 h were 2.0 (95% CI 1.5, 2.7) and 0.53 (95% CI 0.27, 1) for stillbirths, 1.5 (95% CI 0.94, 2.4) and 0.73 (95% CI 0.39, 1.4) for neonatal deaths and 1.5 (95% 1, 2) and 0.78 (95% 0.64, 0.95) for Neonatal Intensive Care Unit admission. Conclusion Measure of proteinuria is a poor predictor of either maternal or fetal complications in women with pre-eclampsia